Technical Field
Embodiments described herein relate to storage systems, and more particularly, to techniques for optimistically managing compressed data in storage systems based on data mapping into small fixed-sized blocks.
Description of the Related Art
Managing compressed data in a fixed-size block storage system presents multiple challenges. Input data streams can be partitioned into equal-sized blocks, but compressing the equal-sized input blocks results in compressed buffers of arbitrary size, which do not fit nicely into the fixed-size physical locations on the target storage devices. Also, combining compression with deduplication can increase the complexity of the storage system. To reduce metadata overhead, the input block size used for partitioning the input data stream could be increased, but this would have a negative effect on the deduplication ratio, as deduplication is typically implemented by block hashing. On the other hand, compressing small input blocks is typically sub-optimal because compression algorithms tend to be less efficient on smaller sized input data.